Parking assistance apparatus, vehicle-side apparatus of parking assistance apparatus, parking assist method, and parking assist program

ABSTRACT

A parking assistance apparatus makes it easy for a driver to grasp a relative positional relation between a vehicle and a target parking position and makes it possible to park the vehicle at the target parking position in a parking space with accuracy. 
     An image of a mark M in a parking space S is taken by a vehicle-side camera  7 . A relative position calculation portion  8  calculates a relative positional relation between the vehicle and a target parking position T based on the taken image of the mark M. A backward locus calculation portion  9  calculates an expected locus when the vehicle moves based on the relative positional relation and based on a steering angle detected by a steering angle sensor  15 . An image of the parking space S is taken by a parking-space-side camera  3  and transmitted to a vehicle-side apparatus  1 . An image composition portion  12  composes the image of the parking space S from the parking-space-side camera  3  and the expected locus calculated by the backward locus calculation portion  9 . The composed image is displayed on the monitor  14  located in the vicinity of a drivers seat of the vehicle.

TECHNICAL FIELD

The present invention relates to a parking assistance apparatus and, inparticular, to a parking assistance apparatus which allows a driver tograsp a relative positional relation between a vehicle and a targetparking position when the vehicle is backing up to park.

The present invention also relates to a vehicle-side apparatus of such aparking assistance apparatus.

Further, the present invention relates to a parking assist method forassisting parking in such a manner and a parking assist program forexecution by a computer.

BACKGROUND ART

Conventionally, a steering assistance apparatus has been developed whichassists drive operation by displaying a rear view from a vehicle takenby a camera mounted on the vehicle on a monitor and superimposing on themonitor an expected locus when a vehicle is backing up in response toinformation such as related to steering angle detected by a steeringangle sensor. According to such a steering assistance apparatus, adriver, for example, can park into a parking space by driving thevehicle while viewing the expected locus on a monitor. However, thedriver cannot easily grasp the positional relation between the targetparking space and the vehicle because the rear view taken by the cameramounted on the vehicle changes in the screen of the monitor as thevehicle proceeds.

Therefore, Patent Documents 1-3, for example, disclose a parkingassistance apparatus for taking an image of an area around the entranceof a parking space by a fixed camera mounted at the parking facilityside, transmitting the taken image data to the vehicle side to display adata on the display portion of the vehicle, calculating a leading locusfrom the position of the vehicle to a target parking position in theparking space, and displaying the leading locus superimposed on thetaken image data. The vehicle backs up to the target parking position,and when the vehicle reaches an area around the entrance of the parkingspace, the vehicle is displayed in the image taken by the fixed cameraof the parking facility side. Then the parking is performed by movingthe vehicle along the leading locus. As the image taken by the fixedcamera is fixed regardless of movement of the vehicle, it becomes easierfor the driver to grasp the positional relation between the targetparking position and the vehicle.

[Patent Document 1] JP 2007-148472 A

[Patent Document 2] JP 2007-161118 A

[Patent Document 3] JP 2007-161119 A

DISCLOSURE OF THE INVENTION Problems to be Solved by the Invention

In order to calculate a leading locus from the position of a vehicle toa target parking position as disclosed in Patent Documents 1-3, arelative positional relation between the vehicle and the target parkingposition needs to be identified. However, it is difficult to identifythe position of the vehicle with accuracy by attempting to recognize thevehicle that appears in the image taken by the fixed camera mounted atthe parking facility side because vehicles have respectively differentshapes, sizes, colors, etc. Although license plates attached to eachvehicle have a common rectangular shape and a common size regardless ofthe vehicle, it is difficult to identify the relative positionalrelation between the position of the vehicle and the target parkingposition with accuracy in the case where the contour of the licenseplate cannot be recognized clearly, e.g. where the license plate and thevehicle body around it are of the same color, even if the license plateis used as a clue.

Thus, there is a problem that it is difficult to lead a vehicle to atarget parking position with accuracy by calculating an appropriateleading route.

Also, the apparatus of Patent Document 1 guides a steering angle foraligning the vehicle with the leading route by means of sound or thelike based on a signal received from a steering sensor. However, in thiscase, it is difficult to grasp excess and deficient actual steeringangle with reference to the guided steering angle, so it is difficult topark the vehicle with accuracy even if the leading route is appropriate.

The present invention has been made in light of such conventionalproblems, and has an object to provide a parking assistance apparatusand a vehicle-side apparatus of the parking assistance apparatus whichmake it easier for a driver to grasp the relative positional relationbetween a vehicle and a target parking position and make it possible forthe driver to park the vehicle at the target parking position withaccuracy.

Also, the present invention has an object to provide a parking assistmethod for performing such parking assistance and a parking assistprogram for execution by a computer.

Means for Solving the Problems

A parking assistance apparatus according to the present inventioncomprises a parking-space-side apparatus and a vehicle-side apparatus,wherein:

the parking-space-side apparatus includes: a fixed target that is fixedwith a predetermined positional relation with respect to a targetparking position in a parking space and has at least one characteristicpoint; a parking-space-side camera for taking an image of the parkingspace; and a parking-space-side communication portion for transmittingan image taken by the parking-space-side camera; and

the vehicle-side apparatus includes: a vehicle-side camera for taking animage of the fixed target; a vehicle-side communication portion forreceiving the image transmitted by the parking-space-side communicationportion; a monitor located in the vicinity of a drivers seat; a steeringangle sensor for detecting a steering angle of a vehicle, a relativeposition calculation portion for calculating a relative positionalrelation between the vehicle and the target parking position based on animage of the fixed target taken by the vehicle-side camera; an expectedlocus calculation portion for calculating an expected locus when thevehicle moves based on the steering angle detected by the steering anglesensor; and an image composition portion for displaying on the monitorinformation regarding parking superimposed on a background image,wherein the information regarding parking is obtained based on therelative positional relation between the vehicle and the target parkingposition calculated by the relative position calculation portion andbased on the expected locus calculated by the expected locus calculationportion, and wherein the background image is based on the image receivedat the vehicle-side communication portion.

Also, the parking-space-side apparatus may be constructed so as toinclude a plurality of the fixed targets located at different positions;and the relative position calculation portion calculates the relativepositional relation between the vehicle and the target parking positionbased on any of the images of the fixed targets taken by thevehicle-side camera.

The vehicle-side apparatus may include an image selection portion,wherein the image selection portion: selects an image from thevehicle-side camera if the distance between the vehicle and the targetparking position is larger than a predetermined value; and selects animage from the image composition portion if the distance is less than orequal to the predetermined value, based on the relative positionalrelation between the vehicle and the target parking position calculatedby the relative position calculation portion, and makes the monitordisplay the selected image.

The vehicle-side apparatus may further include a sensor related totravel of the vehicle, and an automatic steering apparatus may create asteering signal for steering the vehicle automatically based on adetection signal from the sensor related to travel of the vehicle andbased on the parking locus calculated by the parking locus calculationportion.

The vehicle-side apparatus may further includes a sensor related totravel of the vehicle, and an automatic traveling apparatus may create atraveling signal for moving the vehicle automatically based on adetection signal from the sensor related to travel of the vehicle andbased on the parking locus calculated by the parking locus calculationportion.

A vehicle-side apparatus of a parking assistance apparatus according tothe present invention comprises: a vehicle-side camera for taking animage of a fixed target;

a vehicle-side communication portion for receiving an image transmittedby a parking-space-side apparatus, the parking-space-side apparatusincluding: a fixed target that is fixed with a predetermined positionalrelation with respect to a target parking position in a parking spaceand has at least one characteristic point; a parking-space-side camerafor taking an image of the parking space; and a parking-space-sidecommunication portion for transmitting an image taken by theparking-space-side camera;

a monitor located in the vicinity of a drivers seat; a steering anglesensor for detecting a steering angle of a vehicle, a relative positioncalculation portion for calculating a relative positional relationbetween the vehicle and the target parking position based on the imageof the fixed target taken by the vehicle-side camera; an expected locuscalculation portion for calculating an expected locus when the vehiclemoves based on the steering angle detected by the steering angle sensor;and an image composition portion for displaying information regardingparking superimposed on a background image, wherein the informationregarding parking is obtained based on the relative positional relationbetween the vehicle and the target parking position calculated by therelative position calculation portion and based on the expected locuscalculated by the expected locus calculation portion, and wherein thebackground image is based on the image received at the vehicle-sidecommunication portion.

A parking assist method according to the present invention comprises thesteps of: taking an image of a parking space by a parking-space-sidecamera; transmitting the image of the parking space taken by theparking-space-side camera to a vehicle side, taking an image of a fixedtarget that is fixed with a predetermined positional relation withrespect to a target parking position in a parking space and has at leastone characteristic point by a vehicle-side camera; calculating arelative positional relation between the vehicle and the target parkingposition based on the image of the fixed target taken by thevehicle-side camera; detecting a steering angle of a vehicle;calculating an expected locus when the vehicle moves based on thedetected steering angle; and displaying on the monitor informationregarding parking superimposed on a background image, wherein theinformation regarding parking is obtained based on the calculatedrelative positional relation between the vehicle and the target parkingposition and based on the calculated expected locus, and wherein thebackground image is based on the image of the parking space taken by thevehicle-side camera.

A parking assist program according to the present invention is forexecuting the steps of: taking an image of a parking space by aparking-space-side camera; transmitting the image of the parking spacetaken by the parking-space-side camera to vehicle side, taking an imageof a fixed target that is fixed with a predetermined positional relationwith respect to a target parking position in a parking space and has atleast one characteristic point by a vehicle-side camera; calculating arelative positional relation between the vehicle and the target parkingposition based on the image of the fixed target taken by thevehicle-side camera; detecting a steering angle of a vehicle;calculating an expected locus when the vehicle moves based on thedetected steering angle; and displaying information regarding parkingsuperimposed on a background image, wherein the information regardingparking is obtained based on the calculated relative positional relationbetween the vehicle and the target parking position and based on thecalculated expected locus, and wherein the background image of theparking space is based on the image taken by the vehicle-side camera.

EFFECTS OF THE INVENTION

According to the present invention, a driver can grasp the relativepositional relation between the vehicle and the target parking positioneasily and it becomes possible for the driver to park the vehicle at thetarget parking position in the parking space with accuracy.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a construction of a parking assistanceapparatus according to a first embodiment of the present invention;

FIG. 2 is a block diagram showing a construction of a relative positioncalculation portion in the first embodiment;

FIG. 3 shows a mark used in the first embodiment;

FIG. 4 is a flowchart showing an operation of the first embodiment;

FIG. 5 is a plan view showing a positional relation between a parkingspace and a vehicle in the first embodiment;

FIG. 6 is a flowchart showing an operation in a step for calculating arelative positional relation between the vehicle and the target parkingposition in the first embodiment;

FIG. 7 shows a screen of a monitor in the first embodiment;

FIG. 8 shows a screen of a monitor in the first embodiment;

FIG. 9 is a block diagram showing a construction of a parking assistanceapparatus according to a second embodiment;

FIG. 10 shows a screen of a monitor in the second embodiment;

FIG. 11 shows a screen of a monitor in a third embodiment;

FIG. 12 is a block diagram showing a construction of a parkingassistance apparatus according to a fourth embodiment;

FIG. 13 shows a screen of a monitor in the fourth embodiment;

FIG. 14 is a block diagram showing a construction of a parkingassistance apparatus according to a fifth embodiment;

FIG. 15 is a block diagram showing a construction of a parkingassistance apparatus according to a sixth embodiment;

FIG. 16 is a block diagram showing a construction of a parkingassistance apparatus according to a seventh embodiment;

FIG. 17 is a plan view showing a parking space and a mark in a eighthembodiment;

FIG. 18 is a plan view showing a relation between the parking space andthe vehicle in parking operation in the eighth embodiment;

FIG. 19 shows a parking space where a mark used in an alternativeexample of the eighth embodiment is located;

FIG. 20 shows a mark used in a ninth embodiment;

FIG. 21 is a block diagram showing a construction of an apparatus fordisplaying a mark in a tenth embodiment;

FIG. 22 is a perspective view showing how a mark is displayed using aprojector;

FIG. 23 is a perspective view showing how a mark is displayed using alaser scanner;

FIG. 24 is a plan view showing how a mark is displayed using multipleilluminators;

FIG. 25 is a plan view showing how a mark is displayed using anillumination device in a form such as an electric billboard;

FIG. 26 is a plan view showing a parking space where aparking-space-side camera used in an eleventh embodiment is located;

FIG. 27 is a block diagram showing a construction of a parkingassistance apparatus according to a twelfth embodiment;

FIG. 28 is a block diagram showing a construction of a parkingassistance apparatus according to a thirteenth embodiment;

FIG. 29 is a flowchart showing an operation of the thirteenthembodiment;

FIG. 30 is a block diagram showing a construction of a parkingassistance apparatus according to a fourteenth embodiment;

FIG. 31 is a flowchart showing an operation of the fourteenthembodiment;

FIG. 32 is a block diagram showing a construction of a parkingassistance apparatus according to a fifteenth embodiment;

FIG. 33 is a block diagram showing a construction of a relative positioncalculation portion in the fifteenth embodiment; and

FIG. 34 is a plan view showing a positional relation between a parkingspace and a vehicle in the fifteenth embodiment.

BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, embodiments of the present invention will be describedbased on the accompanying drawings.

First Embodiment

The construction of a parking assistance apparatus according to a firstembodiment of the present invention is shown in FIG. 1. The parkingassistance apparatus consists of a parking-space-side apparatus 1located in or around a parking space S such as a garage and avehicle-side apparatus 2 mounted on a vehicle which is to be parked inthe parking space S.

The parking-space-side apparatus 1 has a parking-space-side camera 3located at an inner side of the parking space S for taking an image ofan area around an entrance of the parking space S. A communicationportion (a parking-space-side communication portion) 5 is connected tothe parking-space-side camera 3 via an encoder 4. The encoder 4 is forcompressing the image taken by the parking-space-side camera 3 into aformat suitable for wireless transmission. The communication portion 5is a portion for transmitting the image data compressed by the encoder 4to the vehicle-side apparatus 2. A control portion 6 is connected to theparking-space-side camera 3 and the communication portion 5. Further,the parking-space-side apparatus 1 has a mark M (fixed target) installedon a floor surface around the entrance of the parking space S.

It is assumed that internal parameters (focal length, distortionconstant, etc.) and external parameters (relative position, angle, etc.with reference to the parking space S) of the parking-space-side camera3 are known in advance. Similarly, it is assumed that a relativeposition of the mark M with reference to a target parking position T inthe parking space S is known in advance.

On the other hand, the vehicle-side apparatus 2 has a vehicle-sidecamera 7 mounted on a rear portion of the vehicle for taking an image ofa rear view from the vehicle. An image of the mark M in the parkingspace S is taken by the vehicle-side camera 7 when backing into theparking space S is performed. A relative position calculation portion 8is connected to the vehicle-side camera 7 and a backward locuscalculation portion 9 is connected to the relative position calculationportion 8. The backward locus calculation portion 9 functions as anexpected locus calculation portion in the first embodiment. Also, thevehicle-side apparatus 2 has a communication portion 10 (vehicle-sidecommunication portion) which communicates with the communication portion5 of the parking-space-side apparatus 1. A decoder 11 is connected tothe communication portion 10. The decoder 11 is for decoding thecompressed image data from the parking-space-side apparatus 1 receivedat the communication portion 10.

An image composition portion 12 is connected to the backward locuscalculation portion 9 and the decoder 11. An image selection portion 13is connected to both the image composition portion 12 and thevehicle-side camera 7. A monitor 14 located in the vicinity of thedrivers seat of the vehicle is connected to the image selection portion13.

Also, a steering angle sensor 15 for detecting a steering angle of thevehicle is connected to the backward locus calculation portion 9.

Further, a control portion 16 is connected to the vehicle-side camera 7,the relative position calculation portion 8 and the communicationportion 10.

It is assumed that internal parameters (focal length, distortionconstant, etc.) and a relative position, an angle, etc. of thevehicle-side camera 7 with reference to the vehicle V are known inadvance.

As shown in FIG. 2, the relative position calculation portion 8 of thevehicle-side apparatus 2 consists of an image processing means 17, apositional parameter calculation means 18 and a relative positionidentification means 19 connected serially in this order between thevehicle-side camera 7 and the backward locus calculation portion 9.

Also, the mark M of the parking-space-side apparatus 1 is installed at apredetermined position having a predetermined positional relation withreference to the parking space S, and it is assumed that thepredetermined positional relation of the mark M with reference to theparking space S is grasped in advance. As this mark M, for instance, asshown in FIG. 3, it is possible to use a figure having an external formin a square shape in which four isosceles right-angled triangles areabutted against each other. Each isosceles right-angled triangle isgiven a color that is different from those of its adjacent isoscelesright-angled triangles and this mark M has five characteristic points C1to C5 formed by multiple side intersections.

Next, operation of the first embodiment will be described with respectto the flowchart of FIG. 4.

First, in Step S1, as shown in FIG. 5, in a state in which the vehicle Vis positioned at a location A in the vicinity of the parking space Swith the mark M in the field of view of the vehicle-side camera 7, thevehicle-side camera 7 is operated by the control portion 16 of thevehicle-side apparatus 2 to take an image of the mark M.

The image taken by the vehicle-side camera 7 is inputted to the relativeposition calculation portion 8. In subsequent Step S2, the relativeposition calculation portion 8 calculates a relative positional relationbetween the vehicle V and the parking space S. Upon this, inside therelative position calculation portion 8, the calculation of the relativepositional relation is performed according to Steps S12-S14 shown inFIG. 6 as described below.

That is, in Step S12, the image processing means 17 of the relativeposition calculation portion 8 extracts the five characteristic pointsC1 to C5 of the mark M from the image of the mark M taken by thevehicle-side camera 7 and recognizes and obtains each of two-dimensionalcoordinates of those characteristic points C1 to C5 on the image.

Next, in Step S13, based on the two-dimensional coordinates of each ofthe characteristic points C1 to C5 recognized by the image processingmeans 17, the positional parameter calculation means 18 calculatespositional parameters including six parameters that arethree-dimensional coordinates (x, y, z), a tilt angle (dip angle), a panangle (direction angle) and a swing angle (rotation angle) of thevehicle-side camera 7 with reference to the mark M.

Here, a positional parameter calculation method by the positionalparameter calculation means 18 will be described.

First, a point on the ground dropped from a center of a rear axle of thevehicle V vertically with respect to a road surface is set as an originO, a road surface coordinate system is assumed in which an x axis and ay axis are set in a horizontal direction and a z axis is set in avertical direction, and an image coordinate system is assumed in whichan X axis and a Y axis are set on the image taken by the vehicle-sidecamera 7.

Coordinate values Xm and Ym (m=1 to 5) of the characteristic points C1to C5 of the mark M in the image coordinate system are expressed by thefollowing expressions from the six positional parameters of thecharacteristic points C1 to C5 of the mark M in the road surfacecoordinate system, in other words, coordinate values xm, ym and zm, andangle parameters Kn (n=1 to 3) that are the tilt angle (dip angle), thepan angle (direction angle) and the swing angle (rotation angle)described above using functions F and G.

Xm=F(xm,ym,zm,Kn)+DXm

Ym=G(xm,ym,zm,Kn)+DYm

Here, DXm and DYm are deviations between the X coordinates and the Ycoordinates of the characteristic points C1 to C5 calculated using thefunctions F and G, and the coordinate values Xm and Ym of thecharacteristic points C1 to C5 recognized by the image processing means17.

In other words, through expression of each of the X coordinates and theY coordinates of the five characteristic points C1 to C5, ten relationalexpressions are created in total with respect to the six positionalparameters (xm, ym, zm, Kn).

Therefore, the positional parameters (xm, ym, zm, Kn) are obtained whichminimizes the following sum of squares of the deviations DXm and DYm.

S=Σ(DXm²+DYm²)

In other words, an optimization problem that minimizes S is solved. Itis possible to use a known optimization method such as a simplex method,a steepest descent method, a Newton method, or a quasi-Newton method.

It should be noted here that by creating relational expressions whosenumber is greater than the number “six” of the positional parameters(xm, ym, zm, Kn) to be calculated, the positional parameters aredetermined, so it becomes possible to obtain the positional parameters(xm, ym, zm, Kn) with accuracy.

In the first embodiment of the present invention, ten relationalexpressions are created for the six positional parameters (xm, ym, zm,Kn) from the five characteristic points C1 to C5, but it is sufficientthat the number of the relational expressions is equal to or greaterthan the number of the positional parameters (xm, ym, zm, Kn) to becalculated and when six relational expressions are created from at leastthree characteristic points, it is possible to calculate six positionalparameters (xm, ym, zm, Kn).

In Step S14, using the positional parameters of the vehicle-side camera7 thus calculated, the relative position identification means 19identifies a relative positional relation between the vehicle V and thetarget parking position T in the parking space S. That is, the relativepositional relation between the vehicle-side camera 7 and the targetparking position T is identified based on the positional parameterscalculated by the positional parameter calculation means 18 and based onthe predetermined positional relation between the mark M and the targetposition T in the parking space S grasped in advance and, further, therelative positional relation between the vehicle V and the targetparking position T is identified because the predetermined positionalrelation between the vehicle-side camera 7 and the vehicle V is graspedin advance.

The relative positional relation between the vehicle V and the targetparking position T thus calculated by the relative position calculationportion 8 is sent to the backward locus calculation portion 9. In StepS3 in the flowchart of FIG. 4, the backward locus calculation portion 9is inputted a steering angle signal from the steering angle sensor 15and calculates an expected backward locus with reference to a roadsurface for the case the vehicle V travels backward based on therelative positional relation between the vehicle V and the parking spaceS and based on a current steering angle obtained from the steering anglesignal. The expected backward locus is an expected locus when thevehicle moves in the first embodiment. In the concrete, drawing data fordrawing an expected backward locus on a road surface behind the vehicleis calculated by the backward locus calculation portion 9.

Next, in Step S4, the control portion 16 transmits a request signal forimage data from the communication portion 10 to the parking-space-sideapparatus 1.

At the parking-space-side apparatus 1, upon reception of the requestsignal for image data from the vehicle-side apparatus 2 by communicationportion 5, the parking-space-side camera 3 is operated by the controlportion 6 and an image of an area around the entrance of the parkingspace S is taken in Step S5. Then, the image data taken by theparking-space-side camera 3 is transmitted from the communicationportion 5 to the vehicle-side apparatus 2 after being compressed by theencoder 4 into a form suitable for wireless transmission.

In Step S6, upon reception of the image data from the parking-space-sideapparatus 1 by the communication portion 10 of the vehicle-sideapparatus 2, the image data is sent to the image composition portion 12after being decoded by the decoder 11.

In Step S7, the image composition portion 12 transforms the drawing dataof the expected backward locus with reference to the road surfacecalculated by the backward locus calculation portion 9 based on theparameters of the parking-space-side camera 3 into drawing data in acoordinate system for an imaging screen of the parking-space-side camera3. The image composition portion 12 then composes the transformeddrawing data into the image data from the parking-space-side camera 3sent by the decoder 11. As a result, composed data is created in whichthe expected backward locus of the vehicle V is superimposed on abackground image of the area around the entrance of the parking space Staken by the parking-space-side camera 3.

The composed data created by the image composition portion 12 in thismanner and the image data of the rear view from the vehicle taken by thevehicle-side camera 7 are inputted to the image selection portion 13.

Here, in Step S8, the control portion 16 compares a distance L betweenthe current position of the vehicle V and the target parking position Twith a predetermined value Lth based on the relative positional relationbetween the vehicle V and the target parking position T calculated bythe relative positional relation calculation portion 8. If the distanceL is larger than the predetermined value Lth, the control portion 16makes the image selection portion 13 select the image data from thevehicle-side camera 7 in Step S9. As a result, the image of the rearview from the vehicle taken by the vehicle-side camera 7 is displayed onthe screen of the monitor 14 in the vicinity of the drivers seat.

On the other hand, if the distance L between the current position of thevehicle V and the target parking position T is equal to or less than thepredetermined value Lth, the control portion 16 makes the imageselection portion 13 select the composed data from the image compositionportion 12 in Step S10. As a result, the background image of the areaaround the entrance of the parking space S taken by theparking-space-side camera 3 and the expected backward locus of thevehicle V are superimposed and displayed on the screen of the monitor14.

Then, it is determined in Step S11 whether to end the parkingassistance. Steps S1-S10 are executed repeatedly until the vehicle Vreaches the target parking position T and it is determined that theparking assistance should end.

It should be noted that the predetermined length Lth used in Step S8 maybe set based on, for example as shown in FIG. 5, the distance betweenthe position of the vehicle V and the target parking position T at thetime a portion of the vehicle V comes into the field of view of theparking-space-side camera 3, assuming a garage where both side and aninner side of the parking space S are defined by walls or the like.

If the predetermined length Lth is set in this manner, when the vehicleis not in the field of view of the parking-space-side camera 3 as shownby vehicle position A in FIG. 5, the image of the rear view from thevehicle taken by the vehicle-side camera 7 is displayed on the screen ofthe monitor 14. After the vehicle V enters the field of view of theparking-space-side camera 3 as shown by the vehicle position B, abackground image of the area around the entrance of the parking space Sincluding at least a portion of the vehicle V taken by theparking-space-side camera 3 and the expected backward locus of thevehicle V are displayed on the screen of the monitor 14 superimposed asshown in FIG. 7.

It should be noted that, in FIG. 7, the expected backward locus includesa left-right pair of vehicle width lines E1 and E2 which curve inresponse to a steering angle and line segments E3-E5 representingpredetermined lengths from the rear end of the vehicle, e.g. 0.50 m, 1.0m, and 2.0 m. Further, vehicle width lines E6 and E7 corresponding tostraight backward travel are also superimposed.

The background image of the area around the entrance of the parkingspace S taken by the parking-space-side camera 3 is fixed with referenceto the screen of the monitor 14 regardless of progress of the vehicle V,so only the vehicle V moves in the background image as the vehicle Vproceeds as shown in FIG. 8. This makes it easier for the driver tograsp the relative positional relation between the vehicle V and thetarget parking position T by viewing the vehicle, the parking space Sand the expected backward locus displayed on the screen of the monitor14. Therefore, the driver can park the vehicle V at the target parkingposition T in the parking space S with accuracy.

It should be noted that the relative position calculation portion 8, thebackward locus calculation portion 9, the image composition portion 12,the image selection portion 13 and the control portion 16 of thevehicle-side apparatus 2 and the control portion 6 of theparking-space-side apparatus 1 may each be formed by a computerrespectively. By recording the operations of Steps S1-S10 in FIG. 4 in arecording medium, each step can be executed by the computers.

Further, as the positional parameters consisting of the six parameterscomprising the three-dimensional coordinates (x, y, z), tilt angle (dipangle), pan angle (direction angle) and swing angle (rotation angle) ofthe vehicle-side camera 7 with reference to the mark M in the firstembodiment as mentioned above are calculated, the relative positionalrelation between the mark M and the vehicle V can be identifiedprecisely and parking assistance can be provided with accuracy even ifthere is a gap or inclination between the floor surface of the parkingspace S where the mark M is located and the road surface of the currentposition below the vehicle V.

Nevertheless, if there is no inclination between the floor surface ofthe parking space S where the mark M is located and the road surfacebelow the current position of the vehicle V, the relative positionalrelation between the mark M and the vehicle V can be identified bycalculating positional parameters consisting of at least four parameterscomprising the three-dimensional coordinates (x, y, z) and pan angle(direction angle) of the vehicle-side camera 7 with reference to themark M. In this case, the four positional parameters can be determinedby creating four relational expressions by two-dimensional coordinatesof at least two characteristic points of the mark M. However, it ispreferable to calculate the four positional parameters with accuracyusing a minimum squares method or the like based on two-dimensionalcoordinates of a larger number of characteristic points.

Further, if the mark M and the vehicle V are on the same plane and thereis no gap or inclination between the floor surface of the parking spaceS where the mark M is located and the road surface below the currentposition of the vehicle V, the relative positional relation between themark M and the vehicle V can be identified by calculating positionalparameters consisting of three parameters comprising two-dimensionalcoordinates (x, y) and pan angle (direction angle) of the vehicle-sidecamera 7 with reference to the mark M. Also in this case, the threepositional parameters can be determined by creating four relationalexpressions by two-dimensional coordinates of at least twocharacteristic points of the mark M. However, it is preferable tocalculate the three positional parameters with accuracy using a minimumsquares method or the like based on two-dimensional coordinates of alarger number of characteristic points.

Second Embodiment

The construction of a parking assistance apparatus according to a secondembodiment is shown in FIG. 9. In this parking assistance apparatus, animage from the vehicle-side camera 7 is input to the image compositionportion 12 in the apparatus of the first embodiment shown in FIG. 1. Theimage composition portion 12 composes the image from the vehicle-sidecamera 7 and the image from the parking-space-side camera 3 to generatea perspective image. Also, the image composition portion 12 furthertransforms the drawing data of the expected backward locus calculated bythe backward locus calculation portion 9 into drawing data in thecoordinate system of the perspective image. Further, the imagecomposition portion 12 composes the transformed drawing data with theperspective image data.

An example of composing an image from the vehicle-side camera 7 and animage from the parking-space-side camera 3 and for generation of aperspective image is shown in Japanese Patent Application Laid-open No.3-99952. Note that, in the second embodiment, a relative positionalrelation between the vehicle-side camera 7 and the parking-space-sidecamera 3 has to be determined and reflected in the composition of imagesfrom the cameras and the generation of the perspective image because therelative positional relation between the cameras varies in response tothe current position of the vehicle V.

In this manner, composed data in which the expected backward locus ofthe vehicle V is superimposed on the perspective image is generated. Asshown in FIG. 10, the perspective image with the superimposed expectedbackward locus of the vehicle V may be displayed on the screen of themonitor 14.

In this second embodiment, the perspective image is generated based onboth the image from the vehicle-side camera 7 and the image from theparking-space-side camera 3. This decreases dead angle in comparison tothe case where a perspective image is generated based only on the imagefrom the vehicle-side camera. Also, the resolution of the generatedperspective image can be improved if portions having higher resolution(e.g. positions nearer to respective camera) are selected forcomposition.

Third Embodiment

The first embodiment above switches between: the display of an image forthe rear view from a vehicle taken by the vehicle-side camera 7; and asuperimposed display of the background image of the area around theentrance of the parking space S taken by the parking-space-side camera 3and an expected backward locus of the vehicle V, in response to thedistance L between the current position of the vehicle V and the targetparking position T. However, the construction is not limited to this.For example, as shown in FIG. 11, the screen of the monitor 14 may beseparated into two portions, and the display of the image for the rearview from a vehicle taken by the vehicle-side camera 7 and thesuperimposed display of the background image of the area around theentrance of the parking space S taken by the parking-space-side camera 3and an expected backward locus of the vehicle V may be displayed all thetime regardless of the distance L between the current position of thevehicle V and the target parking position T. Also, they may be displayedconcurrently.

It should be noted that a perspective image on which the expectedbackward locus of the vehicle V is superimposed, such as shown in thesecond embodiment, may be displayed all the time instead of thesuperimposed display of the background image of the area around theentrance of the parking space S taken by the parking-space-side camera 3and an expected backward locus of the vehicle V, together with the imagefor the rear view from the vehicle taken by the vehicle-side camera 7.

Fourth Embodiment

The construction of a parking assistance apparatus related to a fourthembodiment is shown in FIG. 12. In this parking assistance apparatus, asecond image composition portion 20 is connected between thevehicle-side camera 7 and the image selection portion 13 and thebackward locus calculation portion 9 is connected to the second imagecomposition portion 20 in the apparatus of the first embodiment shown inFIG. 1. The backward locus calculation portion 9 outputs drawing data ofan expected backward locus with reference to a road surface to the imagecomposition portion 12. Also, the backward locus calculation portionoutputs the drawing data of the expected backward locus with referenceto the vehicle V to the second image composition portion 20.

The image composition portion 12 transforms the drawing data of theexpected backward locus with reference to the road surface calculated bythe backward locus calculation portion 9 into drawing data in thecoordinate system for an imaging screen of the parking-space-side camera3 based on the parameters of the parking-space-side camera 3. The imagecomposition portion 12 then composes the transformed drawing data withthe image data from the parking-space-side camera 3. As a result,composed data is created in which the expected backward locus of thevehicle V is superimposed on the background image of the area around theentrance of the parking space S taken by the parking-space-side camera3.

On the other hand, the second image composition portion 20 transformsthe drawing data of the expected backward locus with reference to thevehicle V calculated by the backward locus calculation portion 9 intodrawing data in the coordinate system for an imaging screen of thevehicle-side camera 7 based on the parameters of the vehicle-side camera7. The second image composition portion 20 then composes the transformeddrawing data with the image data from the vehicle-side camera 7. As aresult, composed data is created in which the expected backward locus ofthe vehicle V is superimposed on the background image taken by thevehicle-side camera 7.

The composed data created by the image composition portion 12 and thecomposed data created by the second image composition portion 20 createdin this manner are inputted to the image selection portion 13.

The image selection portion 13 switches between: the superimposeddisplay of the background image for the rear view from the vehicle takenby the vehicle-side camera 7 and the expected backward locus of thevehicle V; and the superimposed display of the background image of thearea around the entrance of the parking space S taken by theparking-space-side camera 3 and the expected backward locus of thevehicle V, in response to the distance L between the current position ofthe vehicle V and the target parking position T. Alternatively, as shownin FIG. 13, the screen of the monitor 14 may be separated into twoportions, and a superimposed image of the background image for the rearview from a vehicle taken by the vehicle-side camera 7 and the expectedbackward locus of the vehicle V and a superimposed display of thebackground image of the area around the entrance of the parking space Staken by the parking-space-side camera 3 and the expected backward locusof the vehicle V may be displayed regardless of the distance L betweenthe current position of the vehicle V and the target parking position Tall the time.

It should be noted that, in the fourth embodiment as well, a perspectiveimage on which the expected backward locus of the vehicle V issuperimposed, such as shown in the second embodiment, may be displayedinstead of the superimposed image of the background image of the areaaround the entrance of the parking space S taken by theparking-space-side camera 3 and an expected backward locus of thevehicle V.

Fifth Embodiment

The construction of a parking assistance apparatus related to the fifthembodiment is shown in FIG. 14. In this parking assistance apparatus, aparking travel distance calculation portion 47 is connected between thebackward locus calculation portion 9 and the image composition portion12 in the apparatus of the first embodiment shown in FIG. 1. The parkingtravel distance calculation portion 47 is a portion that calculates atravel distance of the vehicle V from a current position to the targetparking position T based on the relative positional relation between thevehicle V and the target parking position T calculated by the relativeposition calculation portion 8 and based on the expected backward locuscalculated by the backward locus calculation portion 9.

The image composition portion 12 creates composed data in which thetravel distance of the vehicle V calculated by the parking traveldistance calculation portion 47 is superimposed as information regardingparking on the background image of the area around the entrance of theparking space S taken by the parking-space-side camera 3. As a result,the background image of the area around the entrance of the parkingspace S taken by the parking-space-side camera 3 and the travel distancefor reaching the target parking position T can be superimposed on thescreen of the monitor 14.

It should be noted that the image composition portion 12 may beconstructed so that the expected backward locus of the vehicle V is alsodisplayed superimposed together with the background image of the areaaround the entrance of the parking space S taken by theparking-space-side camera 3 and the travel distance for reaching thetarget parking position T.

Sixth Embodiment

The construction of a parking assistance apparatus related to a sixthembodiment is shown in FIG. 15. In this parking assistance apparatus,the relative position calculation portion 8 and the backward locuscalculation portion 9 are connected to the image composition portion 12in the apparatus of first embodiment shown in FIG. 1 respectively andseparately instead of locating the backward locus calculation portion 9between the relative position calculation portion 8 and the imagecomposition portion 12.

The backward locus calculation portion 9 calculates an expected backwardlocus with reference to the vehicle V corresponding to a currentsteering angle based on the steering angle signal from the steeringangle sensor 15. The image composition portion 12 transforms the drawingdata of the expected backward locus with reference to the vehicle Vcalculated by the backward locus calculation portion 9 into drawing datain the coordinate system for the imaging screen of theparking-space-side camera 3, based on the relative positional relationbetween the vehicle V and the target parking position T calculated bythe relative position calculation portion 8 and based on the parametersof the parking-space-side camera 3. The image composition portion 12then composes the transformed drawing data with the image data from theparking-space-side camera 3 sent from the decoder 11. As a result, acomposed data is created in which the expected backward locus of thevehicle V is superimposed on a background image of the area around theentrance of the parking space S taken by the parking-space-side camera3.

In this manner, an operational effect similar to that of the firstembodiment can also be obtained in the construction in which therelative position calculation portion 8 and the backward locuscalculation portion 9 are connected to the image composition portion 12respectively and separately.

Seventh Embodiment

The construction of a parking assistance apparatus related to a seventhembodiment is shown in FIG. 16. In this parking assistance apparatus, inthe apparatus of the first embodiment shown in FIG. 1, a travelable areacalculation portion 48 and an interference determination portion 49 areconnected serially between the relative position calculation portion 8and the image composition portion 12. The travelable area calculationportion 48 identifies an area travelable by the vehicle V based on therelative positional relation between the vehicle V and the targetparking position T or the mark M calculated by the relative positioncalculation portion 8. Also, the interference determination portion 49determines whether there is interference with an obstacle in the casethe vehicle V travels along the expected backward locus based on thetravelable area calculated by the travelable area calculation portion 48and based on the expected backward locus calculated by the backwardlocus calculation portion 9.

It is assumed that the relation between the mark M and the areatravelable by the vehicle V is recognized in advance. That is, bylocating the mark M in response to the area in which the vehicle V cantravel, it can be grasped that the vehicle V can travel in apredetermined area with reference to the mark V, and the travelable areacan be calculated by recognizing the position of the mark M.

The image composition portion 12 creates composed data in which an imagerepresenting whether or not there is interference with an obstacle bycharacters, a mark, etc., determined by the interference determinationportion 49 is superimposed as information regarding parking on thebackground image of the area around the entrance of the parking space Staken by the parking-space-side camera 3. As a result, the backgroundimage of the area around the entrance of the parking space S taken bythe parking-space-side camera 3 and whether or not there is aninterference from an obstacle are superimposed and displayed on thescreen of the monitor 14.

It should be noted that the image composition portion 12 may beconstructed to display the expected backward locus of the vehicle Vsuperimposed together with the background image taken by theparking-space-side camera 3 and whether or not there is an interference.

Further, an image from the parking-space-side camera 3 received by thecommunication portion 10 and decoded by the decoder 11 may be inputtedto the travelable area calculation portion 48, and the travelable areacalculation portion 48 may calculate an area travelable by the vehicle Vby also considering the image from the parking-space-side camera 3.

Eighth Embodiment

In the first embodiment described above, if the mark M is located on afloor surface near the entrance of the parking space S, the mark M canbe perceived and the characteristic points can be recognized more easilybecause the distance between the vehicle V positioned in the vicinity ofthe parking space S and the mark M is shorter. However, the relativepositional relation between the vehicle V and the target parkingposition T would become impossible to calculate as the backward parkingproceeds and the mark M goes out of the field of view of thevehicle-side camera 7. Therefore, as shown in FIG. 17, marks M1 and M2may be located on the floor surface near the entrance and on the floorat an inner side of the parking space S respectively so that either ofthe marks may be used. With this construction, as shown in FIG. 18, if aportion of the vehicle V enters into the parking space S and the mark M1near the entrance of the parking space S goes out of the field of viewof the vehicle-side camera 7, the relative positional relation betweenthe vehicle V and the target parking position T can be calculated bytaking an image of the mark M2 at the inner side of the parking space Sby the vehicle-side camera 7.

Further, as shown in FIG. 19, the mark M2 at the inner side of theparking space S may be located at an inner side wall surface W ratherthan on the floor surface. With this construction, the mark M2 can berecognized by the vehicle-side camera 7 until the point that parking iscompleted.

It should be noted that the marks used in this invention preferably havea particular shape, color, etc., easily distinguishable from shapespresent in the natural environment, so that their existence can beperceived easily in the image recognition by the image processing means17 and the characteristic points included therein can be easilyrecognized.

Also, it is preferable that the marks have a sufficient size and arelocated at places where they can be easily perceived from the vehicle Vso that the accuracy of the relative positional relation between thevehicle V and the marks calculated based on the two-dimensionalcoordinates of the recognized characteristic points and the accuracy ofthe parking locus calculated based on the relative positional relationcan achieve an aimed parking accuracy.

Concretely, a mark may be located by painting it directly at apredetermined location on a floor surface, a wall surface, etc., of theparking space S. Alternatively, a mark may be located by attaching asheet on which a mark is drawn to a predetermined position.

Ninth Embodiment

In the first to eighth embodiments described above, the figure having anexternal form in a square shape in which four isosceles right-angledtriangles are abutted against each other is used as the marks M, M1 andM2 of the parking space S. However, the marks are not limited to thisand various marks, for example those shown in FIG. 20, may be used.

A mark M3 shown in FIG. 20A has a shape in which two triangles of themark M shown in FIG. 3 are elongated in a predetermined direction d. Themark M3 comprises five characteristic points C1 to C5. If the mark M3 islocated on the floor of the parking space S with the direction ddirected toward the entrance of the parking space S, an image of agenerally square shape can be obtained in a perspective image when thevehicle-side camera 7 takes an image of the mark M3 located obliquelydownward at a point in the vicinity of the parking space S. As a result,extraction of the five characteristic points C1 to C5 from the image ofthe mark M3 becomes easier.

A mark M4 shown in FIG. 20B exchanges positions of the top two trianglesand the bottom two triangles in the mark M shown in FIG. 3. The mark M4comprises seven characteristic points C1 to C7.

A mark M5 shown in FIG. 20C adds two triangles further to the mark Mshown in FIG. 3 and comprises eight characteristic points C1 to C8.

These marks M3 to M5 may be used in a manner similar to the mark M shownin FIG. 3.

Tenth Embodiment

In the first to ninth embodiments described above, the mark used as afixed target may be displayed by means of light. For example, as shownin FIG. 21, a display control apparatus 42 is connected to an opticaldisplay apparatus 41 for displaying the mark M utilizing light, and themark M is displayed at a predetermined location by the optical displayapparatus 41 based on a command from the display control apparatus 42.

For example, as shown in FIG. 22, the mark M may be displayed byprojection using a projector 43 as the optical display apparatus 41.Alternatively, as shown in FIG. 23, the mark M may be displayed byscanning a laser beam using a laser scanner 44 as the optical displayapparatus 41. Further, as shown in FIG. 24, the mark M may be displayedby arranging and fixing multiple illuminators 45 such as LEDs along theshape of the mark M at a predetermined location and making the multipleilluminators 45 illuminate by the display control apparatus 42. Also, asshown in FIG. 25, an illumination apparatus 46 in a form like aso-called electric billboard in which multiple illuminators 45 such asLEDs are filled in a predetermined area may be located in advance, andthe mark M may be displayed by making illuminators 45 within theillumination apparatus 46 illuminate selectively by the display controlapparatus 42. In FIG. 25, only the illuminators 45 painted in blackilluminate and the other illuminators 45 are in a non-illuminatingcondition.

If the mark M is displayed utilizing light as done in the tenthembodiment, there is less risk that the shape of the mark will bedamaged by blurring or wearing out of the surface where the mark islocated compared with the case in which a mark is displayed by paintingor a sheet. Also, the relative positional relation between the vehicle Vand the mark M can be detected with accuracy even after the mark M isused for a long period.

Also, controlling the optical display apparatus 41 by the displaycontrol apparatus 42 makes modification of the display brightness of themark M easier. As a result, by adjusting the brightness in conformitywith the brightness of the surrounding environment, e.g. in the morningor at nighttime, an easily recognizable mark M can be displayed all thetime.

In the case where the projector 43 or the laser scanner 44 is used asthe optical display apparatus 41, the size of the displayed mark M canbe changed easily by controlling the optical display apparatus 41 by thedisplay control apparatus 42. As a result, the accuracy of recognizingthe characteristic points of the mark M can be improved by displaying alarge mark M when the vehicle V is distant from the mark M anddisplaying a small mark M after the vehicle V approaches the mark M. Itshould be noted that, in this case, information regarding the size ofthe mark M has to be communicated to the vehicle-side apparatus 2.

In a similar manner, when the projector 43 or the laser scanner 44 isused as the optical display apparatus 41, the position of the displayedmark M can be changed easily by controlling the optical displayapparatus 41 by the display control apparatus 42. As a result, theposition of the mark M can be changed easily when it is desired toadjust the target parking position in response to the presence of anobstacle or the like in the parking space S so that the vehicle V can beparked at a desired position.

Further, instead of locating a plurality of marks on the floor surfacenear the entrance of the parking space S and on the floor surface at theinner side as shown in FIG. 17, the position of the displayed mark M maybe moved in response to the position of the vehicle V. This saves laborand cost for locating a plurality of marks.

It should be noted that, also when the position of the mark M is changedin this manner, information regarding the position of the mark M mustalso be communicated to the vehicle-side apparatus 2.

Also, in the case that the illumination apparatus 46 in a form such asan electric billboard shown in FIG. 25 is used as the optical displayapparatus 41, the size or position of the mark M described above can bechanged within the area filled by the illuminators 45.

If a projector 43 or laser scanner 44 is used, the color in which themark M is displayed can be changed easily. As a result, by adjusting thedisplay color in conformity with a change of the surroundingenvironment, an easily recognizable mark M can be displayed all thetime.

Also, if a projector 43 or laser scanner 44 is used, the mark M may bedisplayed on a flat surface like a screen located on a floor surface, aside wall surface, etc., of the parking space S. With this construction,the mark M can be displayed without having its shape damaged even if thefloor surface, the side wall surface, etc. are bumpy and the recognitionaccuracy of the characteristic points of the mark M can be improved. Itshould be noted that the flat surface like a screen can be realized byselecting a material or a shape suitable for the location where it isprovided, e.g. by sticking a flexible screen on a surface where it islocated or by providing a flat member.

The intensity, wavelength (a color), etc., of the displaying light ofthe mark M can be modulated by controlling the optical display apparatus41 by the display control apparatus 42, and the image of the mark Mtaken by the camera of the vehicle V can be demodulated. With thisconstruction, influence from noise from sunlight, lighting, etc., can beexcluded and the positions of the characteristic points of the mark Mcan be recognized with high accuracy. Also, by the modulation of thedisplaying light of the mark M, not only the characteristic points ofthe mark M but also various information, such as information regardingthe parking space S itself and/or information regarding a method forparking into the parking space S, can be superimposed. For example,information indicating that the mark M is a passing point toward thetarget parking position or information indicating that the mark M is aparking completion position can be superimposed while changing theposition where the mark M is displayed in response to the position ofthe vehicle V.

The displaying light of the mark M may be anything recognizable by thecamera of the vehicle V. Non-visible light such as infrared orultraviolet may be used. The light may be a displaying light with highspeed modulation which cannot be recognized visually. Further, aso-called imprinting of the mark M may be performed by displaying themark M within a very short duration which cannot be recognized visuallyin an image which can be recognized visually. By recognizing the mark Mthus imprinted by the camera of the vehicle V, the relative positionalrelation between the vehicle V and the mark M can be detected. In asimilar manner, the various information described above may be imprintedin an image or in the mark M.

Eleventh Embodiment

In the first to tenth embodiments described above, a singleparking-space-side camera 3 located at the inner side of the parkingspace S captures the image of the area around the entrance of theparking space S. However, a plurality of parking-space-side cameras maybe located to take respective images in the vicinity of the parkingspace S. For example, as shown in FIG. 26, a parking-space-camera forleft side parking 31 and a parking-space-camera for right side parking32 may be located at respective sides near the entrance of the parkingspace S in addition to the parking-space-side camera 3 located at theinner side of the parking space S. One parking-space-side camera 31 maytake an image of the vehicle V when left side parking is performed asindicated by the vehicle V shown by continuous lines while the otherparking-space-side camera 32 may take an image of the vehicle V whenright side parking is performed as indicated by the vehicle V shown bybroken lines.

Upon this, the control portion 16 of the vehicle-side apparatus 2 mayswitch between the parking-space-side camera 31, or theparking-space-side camera 32, near the entrance of the parking space Sand the parking-space-side camera 3 at the inner side of the parkingspace S by commanding the parking-space-side apparatus 1 via thecommunication portion 10 based on the relative positional relationbetween the vehicle V and the target parking position T calculated bythe relative position calculation portion 8.

That is, when the distance between the vehicle V and the target parkingposition T is large and the vehicle V does not yet enter in the field ofview of the parking-space-side camera 3 at the inner side of the parkingspace S, the control portion 16 of the vehicle-side apparatus 2 receivesthe image data from the parking-space-side camera 31 or 32 by commandingthe parking-space-apparatus 1 via the communication portion 10. Then thecontrol portion 16 of the vehicle-side apparatus 2 has the imagecomposition portion 12 receive the image data to be composed with thedrawing data of the expected backward locus calculated by the backwardlocus calculation portion 9.

Then, when the distance between the vehicle V and the target parkingposition T gets smaller and the vehicle enters into the field of view ofthe parking-space-side camera 3 at the inner side of the parking spaceS, the control portion 16 of the vehicle-side apparatus 2 receives theimage data from the parking-space-side camera 3 by commanding theparking-space-apparatus 1 via the communication portion 10. Then thecontrol portion 16 of the vehicle-side apparatus 2 has the imagecomposition portion 12 receive the image data to be composed with thedrawing data of the expected backward locus calculated by the backwardlocus calculation portion 9.

With this construction, a driver can proceed with the parking operationviewing a background image fixed with respect to the screen of themonitor 14 regardless of movement of the vehicle V from an initial stageof parking where the vehicle V is still distant from the parking spaceS.

In the eleventh embodiment, the expected backward locus of the vehicle Vis superimposed on an image in the vicinity of the parking space S takenby any of the plurality of the parking-space-side cameras. However, aperspective image may be created by composing the image from thevehicle-side camera 7 and the image from a parking-space-side camera andthe expected backward locus of the vehicle V may be displayed on thescreen of the monitor 14 superimposed on the perspective image, as donein the second embodiment. In this case, the perspective image may becreated based on images from each of the plurality of parking-space-sidecameras and the image from the vehicle-side camera 7. Alternatively, theperspective image may be created based on a selected one of theplurality of the parking-space-side cameras and the image from thevehicle-side camera 7.

Twelfth Embodiment

The construction of a parking assistance apparatus related to thetwelfth embodiment is shown in FIG. 27. In this parking assistanceapparatus, a parking locus calculation portion 21 is connected to therelative position calculation portion 8 and a guide apparatus 22 isconnected to the parking locus calculation portion 21 in the apparatusof the first embodiment shown in FIG. 1.

The parking locus calculation portion 21 calculates a parking locus forleading the vehicle V to the target parking position T based on therelative positional relation between the vehicle V and the targetparking position T calculated by the relative position calculationportion 8. The guide apparatus 22 outputs drive operation guideinformation to the driver of the vehicle V.

The guide apparatus 22 consists of a guide information creation portion23, a guide information output portion 24, a yaw rate sensor 25connected to the guide information creation portion 23 and a vehiclespeed sensor 26.

The guide information creation portion 23 creates drive operation guideinformation for moving the vehicle V along a parking locus based ondetection signals from the steering angle sensor 15, the yaw rate sensor25 and the vehicle speed sensor 26 and based on the parking locuscalculated by the parking locus calculation portion 21.

The guide information output portion 24 outputs the guide informationcreated by the guide information creation portion 23. This may beconstituted by a speaker or a buzzer which communicates the guideinformation via the auditory senses of the driver by voice, alarm, etc.Other than this, a display or a lamp which communicates the guideinformation visually by images, illumination, etc., may be used as theguide information output portion 24. Further, a vibrator or the likewhich communicates the guide information tactually by vibration or thelike may also be used as the guide information output portion 24.

The guide information creation portion 23 takes the steering anglesignal from the steering angle sensor 15, a yaw rate signal from the yawrate sensor 25 and a vehicle speed pulse signal from the vehicle speedsensor 26 repeatedly as the vehicle travels. The guide informationcreation portion 23 calculates a turn radius, a turn angle and a movingdistance of the vehicle V based on those signals. As a result, thedifference of position from the relative position of the vehicle V withrespect to the target portion T calculated by the relative positioncalculation portion 8 is calculated and the current position and thecurrent advancing direction of the vehicle V are identified. The guideinformation creation portion 23 creates the drive operation guideinformation for moving the vehicle V along the parking locus bycomparing the position and the advancing direction of the vehicle Videntified in this manner and the parking locus calculated by theparking locus calculation portion 21.

It should be noted that vehicle travel parameters, such as the turnradius of the vehicle V with respect to steering angle, a gain of theyaw rate sensor 25 and the moving distance per vehicle speed pulse, areset in the guide information creation portion 23 in advance. The turnradius, the turn angle and the moving distance of the vehicle V arecalculated using the steering angle signal, the yaw rate signal, thevehicle speed pulse signal and those vehicle travel parameters.

The drive operation guide information created in this manner isoutputted from the guide information output portion 24 to the driver ofthe vehicle V. As a result, the driver can move the vehicle V along theparking locus and park it at the target parking position T in theparking space S merely by driving in accordance with the guideinformation.

Thirteenth Embodiment

The construction of a parking assistance apparatus related to thethirteenth embodiment is shown in FIG. 28. In this thirteenthembodiment, an automatic steering apparatus 27 is connected to theparking locus calculation portion 21 in the apparatus of the twelfthembodiment shown in FIG. 27 instead of the guide apparatus 22. Theautomatic steering apparatus 27 is an apparatus that creates a steeringsignal so that a steering wheel is automatically moved in accordancewith movement of the vehicle V from brake operation and accelerationoperation by the driver and sends the steering signal out to an electricpower steering apparatus (EPS).

The operation of the eleventh embodiment is shown in a flowchart in FIG.29. After a relative positional relation between the vehicle V and thetarget parking position T is calculated by the relative positioncalculation portion 8 in Step S2, a parking locus L is calculated by theparking locus calculation means 5 in subsequent Step S15, and steeringfor moving the vehicle V along the parking locus is automaticallyperformed by the automatic steering apparatus 27 in Step S16. As aresult, the driver can perform parking into the target parking positionT of the parking space S merely by performing braking and accelerationoperations while paying attention to obstacles or the like on theperiphery of the vehicle V.

Fourteenth Embodiment

The construction of a parking assistance apparatus related to thefourteenth embodiment is shown in FIG. 30. In this fourteenthembodiment, an automatic travel apparatus 28 is connected to the parkinglocus calculation portion 21 in the apparatus of the twelfth embodimentshown in FIG. 27 instead of the guide apparatus 22. The automatic travelapparatus 28 is an apparatus that causes the vehicle V to automaticallytravel by outputting travel signals such as brake control signals,acceleration control signals and shift control signals in addition tosteering signals for moving the steering wheel.

The operation of the twelfth embodiment is shown in a flowchart in FIG.31. After a relative positional relation between the vehicle V and thetarget parking position T is calculated by the relative positioncalculation portion 8 in Step S2, a parking locus L is calculated by theparking locus calculation portion 5 in subsequent Step S15, and thevehicle V is automatically moved along the parking locus by theautomatic travel apparatus 28 in Step S17. As a result, the driver canperform automatic parking into the target parking position T of theparking space S merely by paying attention to obstacles or the like onthe periphery of the vehicle V without performing any drive operationsfor parking.

Fifteenth Embodiment

The construction of a parking assistance apparatus related to thefifteenth embodiment is shown in FIG. 32. This fifteenth embodiment isconstructed for assisting parking by forward movement of the vehicleinstead of parking by backward movement of the vehicle in the apparatusof the first embodiment shown in FIG. 1.

That is, the vehicle-side apparatus 2 has a vehicle-side camera 7′mounted on a front portion of the vehicle for taking an image of a frontview from the vehicle instead of the vehicle-side camera 7 for taking animage of a rear view from the vehicle. An image of the mark M in theparking space S is taken by the vehicle-side camera 7′ when forwardparking into the parking space S is performed. Also, the vehicle-sideapparatus 2 has a relative position calculation portion 8′ instead ofthe relative position calculation portion 8 and has a forward locuscalculation portion 9′ instead of the backward locus calculation portion9. The forward locus calculation portion 9′ functions as the expectedlocus calculation portion in the fifteenth embodiment.

As shown in FIG. 33, the relative position calculation portion 8′ of thevehicle-side apparatus 2 in the fifteenth embodiment has a relativeposition identification means 19′ instead of the relative positionidentification means 19 in the relative position calculation portion 8of the first embodiment shown in FIG. 2. Upon identifying a relativepositional relation between the vehicle V and the target parkingposition T in the parking space S, the relative position identificationmeans 19′ of the fifteenth embodiment shown in FIG. 33 uses thepositional relation that the vehicle-side camera 7′ is mounted on afront portion of the vehicle V while the relative positionidentification means 19 of the first embodiment shown in FIG. 2 uses thepositional relation that the vehicle-side camera 7 is mounted on a rearportion of the vehicle V.

In the fifteenth embodiment, as shown in FIG. 34, in a state in whichthe vehicle V is positioned at a location A in the vicinity of theparking space S with the mark M in the field of view of the vehicle-sidecamera 7′, the vehicle-side camera 7′ is operated by the control portion16 of the vehicle-side apparatus 2 to take an image of the mark M.

In a manner similar to Step S3 in the flowchart of FIG. 4 explained inthe first embodiment, the forward locus calculation portion 9′ isinputted a steering angle signal from the steering angle sensor 15 andcalculates an expected forward locus with reference to a road surfacefor when the vehicle V travels forward based on the relative positionalrelation between the vehicle V and the parking space S and based on acurrent steering angle obtained from the steering angle signal. Theexpected forward locus is an expected locus when the vehicle moves inthe fifteenth embodiment.

According to the construction of the fifteenth embodiment, an assistancesimilar to that obtained in the first embodiment for backward parkingcan be obtained for forward parking.

Note that, in any of the second to the fourteenth embodiments describedabove, the vehicle-side camera 7′, relative position calculation portion8′ and forward locus calculation portion 9′ similar to those of thefifteenth embodiment may be provided instead of the vehicle-side camera7, relative position calculation portion 8 and backward locuscalculation portion 9 so that forward parking is assisted.

Further, the construction of any of the second to the fourteenthembodiments may be combined with the construction of the fifteenthembodiment. That is, the vehicle-side apparatus 2 may comprise thevehicle-side camera 7 for taking a rear view from the vehicle and thevehicle-side camera 7′ for taking a front view from the vehicle,relative position calculation portions 8 and 8′, and backward locuscalculation portion 9 and forward locus calculation portion 9′, and mayassist both backward parking and forward parking.

In this case, the vehicle-side apparatus 2 may also comprise a parkingmode selection switch for selectively indicating either backward parkingmode or forward parking mode, and may be constructed so that the drivercan indicate any mode by manipulating the parking mode selection switch.If the backward parking mode is indicated, the vehicle-side apparatus 2may perform backward parking assistance similar to that of the firstembodiment. If the forward parking mode is indicated, the vehicle-sideapparatus 2 may perform forward parking assistance similar to that ofthe fifteenth embodiment.

Other Embodiments

In each of the embodiments described above, an obstacle sensor such asan ultrasonic sensor may be mounted on the vehicle V and a warning canbe issued or an obstacle avoidance operation can be performed. In thiscase, when the existence of an obstacle on the periphery is recognized,a safer parking assist can be realized.

It is also possible to use an object, such as a wheel stop or a patternof a wall surface of a garage, which originally exists on the peripheryof the parking space as the fixed target instead of installing the markat a predetermined place having a predetermined positional relation withrespect to the parking space. However, it is preferable that theexistence of the object is easy to perceive and characteristic pointsinternally included in the object be easy to recognize.

1. A parking assistance apparatus comprising a parking-space-sideapparatus and a vehicle-side apparatus, characterized in that: theparking-space-side apparatus includes: a fixed target that is fixed witha predetermined positional relation with respect to a target parkingposition in a parking space and has at least one characteristic point; aparking-space-side camera for taking an image of the parking space; anda parking-space-side communication portion for transmitting an imagetaken by the parking-space-side camera; and the vehicle-side apparatusincludes: a vehicle-side camera for taking an image of the fixed target;a vehicle-side communication portion for receiving the image transmittedby the parking-space-side communication portion; a monitor located inthe vicinity of a drivers seat; a steering angle sensor for detecting asteering angle of a vehicle, a relative position calculation portion forcalculating a relative positional relation between the vehicle and thetarget parking position based on an image of the fixed target taken bythe vehicle-side camera; an expected locus calculation portion forcalculating an expected locus when the vehicle moves based on thesteering angle detected by the steering angle sensor; and an imagecomposition portion for displaying on the monitor information regardingparking superimposed on a background image, wherein the informationregarding parking is obtained based on the relative positional relationbetween the vehicle and the target parking position calculated by therelative position calculation portion and based on the expected locuscalculated by the expected locus calculation portion, and wherein thebackground image is based on the image received at the vehicle-sidecommunication portion.
 2. A parking assistance apparatus according toclaim 1, wherein the image composition portion transforms the expectedlocus calculated by the expected locus calculation portion into drawingdata in a coordinate system for an imaging screen of theparking-space-side camera and displays the transformed drawing data asinformation regarding parking.
 3. A parking assistance apparatusaccording to claim 1, wherein the vehicle-side apparatus includes aparking travel distance calculation portion for calculating a traveldistance of the vehicle for reaching the target parking position basedon the relative positional relation between the vehicle and the targetparking position calculated by the relative position calculation portionand based on the expected locus calculated by the expected locuscalculation portion; and the image composition portion displays thetravel distance calculated by the parking travel distance calculationportion as information regarding parking.
 4. A parking assistanceapparatus according to claim 1, wherein the vehicle-side apparatusincludes: travelable area calculation portion for identifying an areatravelable by the vehicle based on the relative positional relationbetween the vehicle and the target parking position calculated by therelative position calculation portion; and interference determinationportion for determining whether there is interference with an obstacle,in the case the vehicle travels along the expected locus, based on thetravelable area calculated by the travelable area calculation portionand based on the expected locus calculated by the expected locuscalculation portion, and the image composition portion displays whetherthere is the interference determined by the interference determinationportion as information regarding parking.
 5. A parking assistanceapparatus according to claim 1, wherein the relative positioncalculation portion includes: image processing means for extracting thecharacteristic point of the fixed target based on the image of the fixedtarget taken by the vehicle-side camera and for recognizing atwo-dimensional coordinate of the characteristic point on the image ofthe fixed target; positional parameter calculation means for calculatingpositional parameters of the vehicle-side camera including at least atwo-dimensional coordinate and a pan angle with reference to the fixedtarget based on two or more sets of the two-dimensional coordinatesrecognized by the image processing means; and relative positionidentification means for identifying a relative positional relationbetween the vehicle and the target parking position based on thepositional parameters of the vehicle-side camera calculated by thepositional parameter calculation means and the predetermined positionalrelation of the fixed target with respect to the target parkingposition.
 6. A parking assistance apparatus according to claim 5,wherein: the fixed target has two or more characteristic points; and theimage processing means recognizes two or more sets of thetwo-dimensional coordinates of the characteristic points based on oneimage of the fixed target taken by the vehicle-side camera.
 7. A parkingassistance apparatus according to claim 6, wherein the fixed targetcomprises a mark of a predetermined shape.
 8. A parking assistanceapparatus according to claim 1, wherein: the parking-space-sideapparatus includes a plurality of the fixed targets located at differentpositions; and the relative position calculation portion calculates therelative positional relation between the vehicle and the target parkingposition based on any of the images of the fixed targets taken by thevehicle-side camera.
 9. A parking assistance apparatus according toclaim 1, wherein the vehicle-side apparatus further includes an imageselection portion, wherein the image selection portion: selects an imagefrom the vehicle-side camera if the distance between the vehicle and thetarget parking position is larger than a predetermined value; andselects an image from the image composition portion if the distance isless than or equal to the predetermined value, based on the relativepositional relation between the vehicle and the target parking positioncalculated by the relative position calculation portion, and makes themonitor display the selected image.
 10. A parking assistance apparatusaccording to claim 1, wherein the vehicle-side apparatus makes themonitor display the image taken by the vehicle-side camera and the imagefrom the image composition portion concurrently.
 11. A parkingassistance apparatus according to claim 10, wherein the vehicle-sideapparatus further includes a second image composition portion fordisplaying on the monitor the expected locus calculated by the expectedlocus calculation portion superimposed on the image taken by thevehicle-side camera.
 12. A parking assistance apparatus according toclaim 1, wherein the parking-space-side apparatus includes a pluralityof parking-space-side cameras located at different locations, and thevehicle-side apparatus includes a control portion for making the imagecomposition portion use the image from a parking-space-side cameraselected among the plurality of parking-space-side cameras in responseto the distance between the vehicle and the target parking positionbased on the relative positional relation between the vehicle and thetarget parking position calculated by the relative position calculationportion.
 13. A parking assistance apparatus according to claim 1,wherein the image composition portion: creates a perspective image bycomposing the image from the vehicle-side camera and the image from theparking-space-side camera; and displays the expected locus calculated bythe expected locus calculation portion superimposed on the perspectiveimage as the background image.
 14. A parking assistance apparatusaccording to claim 1, wherein the vehicle-side apparatus furtherincludes: a parking locus calculation portion for calculating a parkinglocus for leading the vehicle to the target parking position based onthe relative positional relation between the vehicle and the targetparking position calculated by the relative position calculationportion; and a guide apparatus for outputting drive operation guideinformation for traveling along the parking locus calculated by theparking locus calculation portion to a driver of the vehicle.
 15. Aparking assistance apparatus according to claim 14, wherein thevehicle-side apparatus further includes a sensor related to travel ofthe vehicle, and the guide apparatus includes: guide informationcreation portion for creating the drive operation guide informationbased on a detected signal from the sensor related to travel of thevehicle and based on the parking locus calculated by the parking locuscalculation portion; and a guide information output portion foroutputting the guide information created by the guide informationcreation portion.
 16. A parking assistance apparatus according to claim1, wherein the vehicle-side apparatus further comprises: a parking locuscalculation portion for calculating a parking locus for leading thevehicle to the target parking position based on the relative positionalrelation between the vehicle and the target parking position calculatedby the relative position calculation portion; and an automatic steeringapparatus for steering the vehicle automatically in order to travelalong the parking locus calculated by the parking locus calculationportion.
 17. A parking assistance apparatus according to claim 16,wherein the vehicle-side apparatus further includes a sensor related totravel of the vehicle, and the automatic steering apparatus creates asteering signal for steering the vehicle automatically based on adetection signal from the sensor related to travel of the vehicle andbased on the parking locus calculated by the parking locus calculationportion.
 18. A parking assistance apparatus according to claim 1,wherein the vehicle-side apparatus further comprises: a parking locuscalculation portion for calculating a parking locus for leading thevehicle to the target parking position based on the relative positionalrelation between the vehicle and the target parking position calculatedby the relative position calculation portion; and an automatic travelingapparatus for moving the vehicle automatically in order to travel alongthe parking locus calculated by the parking locus calculation portion.19. A parking assistance apparatus according to claim 18, wherein thevehicle-side apparatus further includes a sensor related to travel ofthe vehicle, and the automatic traveling apparatus creates a travelingsignal for moving the vehicle automatically based on a detection signalfrom the sensor related to travel of the vehicle and based on theparking locus calculated by the parking locus calculation portion.
 20. Aparking assistance apparatus according to claim 1, wherein the fixedtarget is displayed by means of light.
 21. A parking assistanceapparatus according to claim 20, wherein the fixed target is displayedby projecting or scanning light.
 22. A parking assistance apparatusaccording to claim 20, wherein the fixed target emits light by itself.23. A parking assistance apparatus according to claim 1, wherein theexpected locus when the vehicle moves is an expected backward locus forwhen the vehicle travels backward, and the expected locus calculationportion is a backward locus calculation portion for calculating theexpected backward locus.
 24. A parking assistance apparatus according toclaim 1, wherein the expected locus when the vehicle moves is anexpected forward locus for when the vehicle travels forward, and theexpected locus calculation portion is a forward locus calculationportion for calculating the expected forward locus.
 25. A vehicle-sideapparatus of a parking assistance apparatus characterized in that itcomprises: a vehicle-side camera for taking an image of a fixed target;a vehicle-side communication portion for receiving an image transmittedby a parking-space-side apparatus, the parking-space-side apparatusincluding: a fixed target that is fixed with a predetermined positionalrelation with respect to a target parking position in a parking spaceand has at least one characteristic point; a parking-space-side camerafor taking an image of the parking space; and a parking-space-sidecommunication portion for transmitting an image taken by theparking-space-side camera; a monitor located in the vicinity of adrivers seat; a steering angle sensor for detecting a steering angle ofa vehicle, a relative position calculation portion for calculating arelative positional relation between the vehicle and the target parkingposition based on the image of the fixed target taken by thevehicle-side camera; an expected locus calculation portion forcalculating an expected locus when the vehicle moves based on thesteering angle detected by the steering angle sensor; and an imagecomposition portion for displaying information regarding parkingsuperimposed on a background image, wherein the information regardingparking is obtained based on the relative positional relation betweenthe vehicle and the target parking position calculated by the relativeposition calculation portion and based on the expected locus calculatedby the expected locus calculation portion, and wherein the backgroundimage is based on the image received at the vehicle-side communicationportion.
 26. A parking assist method characterized in that it comprisesthe steps of: taking an image of a parking space by a parking-space-sidecamera; transmitting the image of the parking space taken by theparking-space-side camera to a vehicle side, taking an image of a fixedtarget that is fixed with a predetermined positional relation withrespect to a target parking position in a parking space and has at leastone characteristic point by a vehicle-side camera; calculating arelative positional relation between the vehicle and the target parkingposition based on the image of the fixed target taken by thevehicle-side camera; detecting a steering angle of a vehicle;calculating an expected locus when the vehicle moves based on thedetected steering angle; and displaying on the monitor informationregarding parking superimposed on a background image, wherein theinformation regarding parking is obtained based on the calculatedrelative positional relation between the vehicle and the target parkingposition and based on the calculated expected locus, and wherein thebackground image is based on the image of the parking space taken by thevehicle-side camera.
 27. A parking assist program characterized in thatit is for executing the steps of: taking an image of a parking space bya parking-space-side camera; transmitting the image of the parking spacetaken by the parking-space-side camera to vehicle side, taking an imageof a fixed target that is fixed with a predetermined positional relationwith respect to a target parking position in a parking space and has atleast one characteristic point by a vehicle-side camera; calculating arelative positional relation between the vehicle and the target parkingposition based on the image of the fixed target taken by thevehicle-side camera; detecting a steering angle of a vehicle;calculating an expected locus when the vehicle moves based on thedetected steering angle; and displaying information regarding parkingsuperimposed on a background image, wherein the information regardingparking is obtained based on the calculated relative positional relationbetween the vehicle and the target parking position and based on thecalculated expected locus, and wherein the background image of theparking space is based on the image taken by the vehicle-side camera.